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Recognizing  the  inability  of  current  telemetry  technology  to  meet  emergent  needs  within  the 
Major  Range  and  Test  Facility  Base,  the  Central  Test  and  Evaluation  Investment  Program 
launched  the  integrated  Network  Enhanced  Telemetry  (iNET)  Project.  iNET  is  taking  a 
systems  engineering  approach  to  defining  a  new  architecture  for flight  test  telemetry.  The  iNET 
architecture  is  the  first  major  change  to  the  underlying  architecture  of  flight  test  telemetry  in 
over  50  years!  Changing  an  architecture  that  has  been  in  place  this  long  could  have  unforeseen 
impacts.  Across  our  ranges,  processes,  procedures,  and  systems  have  characteristics  of  the 
traditional  telemetry  architecture  inherent  in  their  design.  A  careful,  defined,  and  disciplined 
process  is  required  for  assuring  the  processes,  procedures,  and  systems  are  ready  to  accept  iNET 
technology.  As  one  of  the  initial  deployment  sites  for  iNET,  NAVAIR  Pax  River  has  conducted 
a  continuous  process  improvement  project  to  study  the  potential  disruptions  and  mitigations  of 
deploying  this  revolutionary  and  potentially  disruptive  technology.  This  article  describes  the 
study  process,  the  potential  disruptions  identified,  the  results  of  the  risk/failure  mode  effect 
analysis,  and  useful  end  products  developed  to  facilitate  the  safe  deployment  of  iNET  at  the 
Naval  Air  Warfare  Center  Aircraft  Division  at  Patuxent  River,  MD. 
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Real-time  telemetry  is  an  integral  com¬ 
ponent  of  flight  test  scenarios  executed 
on  Department  of  Defense  (DoD) 
Major  Range  and  Test  Facility  Base 
(MRTFB)  ranges.  For  the  last  50  years, 
virtually  aU  real-time  telemetry  has  been  point-to- 
point  one-way  transmission  of  data.  Referred  to  as 
serial  streaming  telemetry  (SST),  data  are  transmitted 
one  way,  from  the  test  article  to  the  remotely  located 
test  team.  The  test  team  evaluates  the  data  in  real-time 
to  ensure  safe  test  execution  and  to  monitor  the 
performance  of  the  test  article.  Data  content  and 
format  of  the  SST  data  stream  are  fixed  in  advance  of 
the  test. 

As  the  complexity  of  weapon  systems  increased,  the 
amount  of  data  collected  onboard  test  articles  began  to 
spiral  upwards.  Flowever,  the  rigidity  of  point-to-point 
telemetry,  coupled  with  limited  spectral  resources. 


severely  limited  the  amount  of  data  that  could  be 
transmitted.  Increasingly,  most  of  the  data  collected 
onboard  a  test  article  were  recorded  vice  being 
transmitted.  In  some  cases  less  than  three  percent  of 
the  data  being  collected  are  transmitted.  Test  engineers 
have  to  wait  until  the  test  article  returns  to  base  to 
retrieve  the  data  recorder  so  that  the  majority  of  the 
data  can  be  downloaded  and  analyzed.  Limited  real¬ 
time  access  to  all  the  data  being  collected  has 
negatively  affected  the  cost  and  schedule  associated 
with  flight  test. 

Considering  the  implications  of  this  trend,  it  became 
clear  that  the  traditional  SST  architecture  needed  to 
change.  The  million  doUar  question  was:  “Change  to 
what?”  The  call  for  change  was  led  by  a  rogue  group  of 
telemetry  engineers  who  advocated  scrapping  SST  and 
changing  to  a  technology  based  on  wireless  networks. 
However,  wholesale  replacement  of  SST  telemetry 
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with  wireless  network  technology  was  also  prohlematic. 
While  telemetry  networks,  via  two-way  connectivity, 
could  allow  near  real-time  access  to  all  the  data 
recorded  onboard  the  test  article,  they  were  ill  suited 
for  time  critical,  no  latency  variance  delivery  of  critical 
safety  data.  Despite  its  faults,  SST  excels  at  the  delivery 
of  time  critical,  no  latency  variance  data.  Slowly 
emerged  the  realization  that  telemetry  networks  and 
traditional  SST  were  not  mutually  exclusive.  In  fact, 
they  complement  each  other;  each  doing  well,  what  the 
other  does  poorly.  The  use  of  wireless  network 
technology  to  enhance  traditional  SST  is  the  basis  of 
the  iNET  architecture. 

The  iNET  architecture  describes  how  a  collection  of 
networks  works  in  harmony  with  SST  to  meet  emergent 
needs  within  the  MRTFB.  This  system  can  be  thought 
of  as  a  “network  of  networks”  consisting  of  the  following: 

•  a  vehicular  network  (to  be  developed  by  iNET) 
on  the  test  article  that  handles  all  onboard  data 
acquisition  functions; 

•  a  wireless  network  (to  be  developed  by  iNET) 
that  provides  network  communications  between 
test  articles  and  between  the  test  articles  and  the 
range  infrastructure; 

•  the  existing  SST  links;  and 

•  the  existing  network  infrastructure  on  DoD  Test 
and  Evaluation  (T8cE)  ranges. 

The  test  and  telemetry  communities  realized  that 
the  capabilities  enabled  by  this  architecture  would  have 
far-reaching  consequences.  Not  only  would  emergent 
requirements  be  more  easily  satisfied,  but  T&E  would 
be  revolutionized.  However,  safely  introducing  a 
revolutionary  capability  into  a  high-risk  environment, 
like  flight  test,  could  prove  problematic.  A  disciplined 
process  was  needed  to  assure  that  the  deployment  of 
iNET  did  not  negatively  affect  cost,  schedule,  or  the 
safety  of  executing  flight  test. 

Independently,  during  the  same  time  period,  the 
Department  of  the  Navy  was  developing  a  disciplined 
set  of  tools  to  improve  productivity.  In  1999,  the  Navy 
Depots  adopted  the  use  of  AIRSpeed  initiatives  to 
increase  productivity  through  process  improvement  by 
using  a  common  set  of  industry-proven  tools.  The  use 
of  AIRSpeed/Lean,  Six  Sigma  (ESS)  tool  sets  has 
proven  to  reduce  lead  times,  remove  waste  (non-value- 
added  cost),  and  reduce  variation. 

In  2004,  NAVAIR,  recognizing  the  positive  impacts 
that  the  Depots  were  experiencing  using  AIRSpeed, 
endorsed  the  utilization  of  NAVAIR  AIRSpeed 
initiatives  across  the  Naval  Air  Warfare  Centers 
(NAWC).  Recognizing  that  the  AIRSpeed  tool  set 
could  be  used  to  design  (as  well  as  improve)  a  process, 
the  “iNET  Deployment  Process”  was  launched.  The 


goal  of  the  project  was  to  utilize  the  AIRSpeed/LSS 
tool  sets  to  solve  the  complex  problem  of  deploying 
iNET  technologies  without  disrupting  existing  infra¬ 
structures,  processes,  and  procedures.  This  project 
became  a  Design  for  Lean  Six  Sigma  effort,  since  a 
process  for  iNET  Deployment  did  not  previously  exist. 

The  INET  AIRSpeed  Project:  ‘‘Chartering 
the  Team" 

In  the  fall  of  2007,  the  iNET  Chief  Architect 
developed  a  charter  for  the  NAVAIR  AIRSpeed/ 
Continuous  Process  Improvement  (CPI)  Team,  with 
the  goal  to  define  a  process/concept  of  operations 
(CONOPS)  for  safe  deployment  of  iNET  capabilities 
(Figure  1). 

In  response  to  the  charter,  the  “iNET  Deployment 
Process”  Team  was  established  to  design  a  process  for 
safe  deployment  of  iNET  at  NAWC  Aircraft  Division 
(NAWCAD).  The  Team  consisted  of  subject-matter 
experts  (SMEs)  representing  all  aspects  of  flight  test 
and  range  operations.  This  talented  group  of  SMEs 
possesses  a  breadth  of  experience  including 

•  flight  test, 

•  airborne  instrumentation, 

•  range  communications, 

•  data  processing  and  display, 

•  radio-frequency  (rf)  systems, 

•  systems  safety, 

•  time  space  position  information,  and 

•  risk  management. 

The  SME  knowledge  base  was  used  to  analyze  the 
planned  deployments  of  iNET  capabilities  for  potential 
disruptions  leading  to  negative  impacts  to  cost,  schedule, 
and  safety  of  flight  testing  at  Patuxent  River,  MD. 

Project  execution 

Recognizing  the  potential  of  iNET  technology  to 
revolutionize  flight  test  at  NAWCAD,  this  team  was 
extremely  motivated  and  dedicated  to  the  task.  In 
addition  to  meeting  weekly  for  9  months,  they  met  in 
several  all-day  off-site  meetings.  Using  the  Lean  Six 
Sigma  tool  set,  the  Team  methodically  progressed 
through  the  five  design  steps  for  Lean  Six  Sigma: 
Define,  Measure,  Explore,  Develop,  Implement 
(DMEDI).  While  working  through  the  details  of  the 
process,  the  Team  never  lost  its  focus — to  create  a 
process  for  the  safe  deployment  of  iNET  technology  at 
NAWCAD  Patuxent  River. 

Define 

In  this  phase,  the  goal  is  to  clearly  define  the 
problem.  The  first  step  was  to  validate  the  project 
charter.  Once  the  charter  was  validated,  the  Team  used 
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Project  Name:  I  NET  Deployment  process  Date:  1 1-14-07 

Competency/PEO:  5.2  National  Ranges  Deployment  Champion:  Kathy  Seals 

Project  Sponsor:  Dan  Skelley  5.2.0  Black/Green  Belt:  Brian  Anderson 


Business  Impact  ($) 

Telemetry  systems  have  not  changed  in  roughly  50  years.  The  current  systems  are  very  static.  Many  of 
the  processes  that  govern  flight  testing  have  this  static  nature  embedded.  INET  deployment  offers  2 
way  dynamic  connectivity  for  interfacing  with  data  on  the  fly  however,  it  may  be  a  disruptive  technology 
that  could  have  unforeseen  safety  of  flight  test  impacts,  resulting  in  (x)  $  loss  In  program  operating  cost 
across  the  NAE. _ 

Opportunity  or  Problem  Statement 

When:  Current  and  ongoing  for  the  past  50  years,  the  basic  telemetry  structure  has  not  changed. 

■  iNET  may  yield  disruption  in  tech  demo  deployment  FY09  timeframe. 

■  iNET  may  yield  disruption  in  the  FY1 1/12  Block  (1 )  deployment  timeframe. 

What:  A  process  for  Safe  deployment  of  iNET  for  flight  testing  is  required. 

Where:  Initial  Operation  Capability  (IOC)  -  Patuxent  River  Md.  &  Edwards  AFB 

■  Independent  substantiations  PAX  &  Edwards 

Extent:  1 00%  percent  of  the  time  there  is  not  a  documented  CONORS  process  for  deployment  of 
potentially  disruptive  INET  technologies. 

■  A  CONORS  for  INET  utilization  could  potentially  prevent  loss  of  Test  Article  or  loss  of  Life. 

Goal  Statement 

Define  a  Process  /  CONORS  for  Safe  Deployment  of  iNET  Capabilities. 


Project  Scope 

In  Scope:  All  Flight  Test  Processes  relevant  to  Block  (1)  deployment  of  INET. 

Out  of  scope:  Non-flight  test  processes.  Block  (2)  deployment  of  iNET  and  beyond. 


Project  Plan 

Team  Launch  -  October  30,  2007 
Define  -  November  19,  2007 
Measure  -  March  4,  2008 
Explore  -March  04,  2008 
Develop  -  May  14,  2008 
Implement  -  June  25,  2008 

Validate  -  April  29,  2009 _ 

Team  Selection 

1.  Mark  Smedley- (Airborne  Instrumentation)  SME  /  Project  Green  Belt 

2.  Sid  Jones  -  (iNET  /  RCC)  SME 

3.  Ray  Faulstich  -  (iNET)  SME 

4.  Jonathon  Norton  /Dennis  Normyle  /  Bob  Myers  -  (RTPS  Telemetry)  SME 

5.  Jim  Pilkerton  /_Bob  Craft- (RTPS  Test  Communications)  SME 

6.  Patricia  Khatiblou  /Tom  McCaughey  -  (RTPS  Test  Management /Control  Room)  SME 

7.  Robert  Jacob  /  Jason  Stewart  -  (Range  Safety)  SME 

8.  Paul  Conigliaro/ Jackie  White -(5.1  Flight  Test  Engineering)  SME 

9.  External  -  5.1  &  5.2  West  Coast  SME's 


Figure  1.  NAVAIR  AIRSpeed  Lean  Six  Sigma  Project  Charter:  iNET  Depioyment  Process. 
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tools  such  as  swim  lane  process  mapping  and  voice  of 
the  customer/business  evaluations  to  ensure  that  the 
problem  was  understood.  Ultimately,  the  Team 
thoroughly  reviewed  the  iNET  deployment  schedule, 
future  iNET  capabilities  and  scenarios,  and  scenarios 
relevant  to  deployments  at  Patuxent  River. 

Measure 

In  the  measure  phase,  the  Team  focused  on  creating 
a  project  schedule,  determining  the  project  execution 
strategy  by  using  a  system  engineering  “V”  diagram, 
and  identifying  the  metrics  to  decompose.  The  metrics 
of  choice  were  53  preexisting  process  maps  referred  to 
as  Operational  Sequence  Diagrams  (OSD).  The  OSDs 
were  generated  by  the  iNET  project  during  the  initial 
system  engineering  effort  to  define  the  architecture. 
They  are  diagrams  depicting  the  flow  of  events 
necessary  to  accomplish  each  of  the  53  use  cases  the 
iNET  project  is  chartered  to  meet. 

Erom  these  53,  the  Team  identified  the  predomi¬ 
nant  capability  to  be  first  used  at  NAWCAD.  The 
“Eetch  Data”  scenario  describes  the  details  perceived  as 
needed  to  access  previously  recorded  data  from  the  test 
article  in  real-time.  The  Team  then  began  a  10-week 
brainstorming  session,  meeting  3  hours  per  week,  to 
surface  potential  disruptions  associated  with  deploying 
this  number-one  capability.  Using  the  OSD  for  “Eetch 
Data  Erom  Aircraft,”  the  Team  walked  through  the 
multiple  process  steps  asking  questions  along  the  way, 
such  as,  “If  this  step  were  deployed  now,  what  would 
be  the  impacts  and  or  disruptions  to  existing  processes, 
procedures,  and  infrastructure?” 

Erom  this  lengthy  brainstorming  process,  the  Team 
categorized  144  disruptions  in  a  Eailure  Modes  and 
Effects  Analysis  (EMEA)  spreadsheet. 

Explore 

While  this  method  for  analyzing  the  impacts  of 
deploying  a  capability  was  successful,  it  was  extremely 
labor  intensive.  As  such,  it  was  unsuitable  for  use  as  a 
standard  process  in  deploying  iNET.  The  Team 
determined  that  they  needed  a  standard  process, 
defined  by  a  guidebook,  which  would  steer  the  range 
in  finding  and  mitigating  the  potential  disruptions  in 
the  deployment  of  iNET.  During  this  phase,  the 
concept  of  the  Disruption  Einder  Guidebook  was 
established.  However,  the  process  this  guidebook 
would  describe  was  stiU  a  mystery. 

Develop 

In  the  develop  phase,  the  Team  looked  at  many 
different  approaches  to  identifying  the  potential 
disruptions.  The  lengthy  brainstorming  session  that 
led  to  the  144  potential  disruptions  was  simply  not 
efficient  enough.  A  particularly  creative  member  of  the 


Team  suggested  that  a  process  used  by  industry  to 
uncover  threats  (or  disruptions)  to  the  deployment  of 
complex  software  systems  might  be  of  use.  After  a  bit 
of  research,  it  was  determined  that  a  concept  used  by 
Microsoft  Corporation,  called  “Threat  Modeling,” 
could  be  adapted  to  speed  the  process  of  identifying 
potential  disruptions.  Central  to  this  concept  is  the  use 
of  data-flow  diagrams  (similar  to  the  OSDs  mentioned 
above)  and  an  acronym  to  focus  brainstorming 
disruptions.  In  the  case  of  Microsoft  Corporation, 
they  use  the  STRIDE  acronym  which  stands  for 
Spoofing,  Tampering,  Repudiation,  Information  Dis¬ 
closure,  Denial  of  Service,  and  Elevation  of  Privilege. 

The  first  step  is  to  develop  a  data  flow  diagram  for 
the  capability  being  deployed.  The  Team  found  that  a 
data-flow  diagram  could  quickly  be  created  using  the 
OSDs  as  a  launching  point. 

Once  a  data-flow  diagram  is  documented,  the  next 
step  is  to  use  the  categories  identified  in  the  acronym  (in 
the  Microsoft  case,  STRIDE)  to  guide  the  Team 
through  a  shortened  brainstorming  exercise  to  identify 
the  potential  disruptions.  However,  while  the  STRIDE 
set  of  threats  might  work  well  for  the  deployment  of  a 
complex  software  system  like  Microsoft  Vista,  it  was  iU 
suited  for  analysis  of  deploying  iNET  technology.  The 
Team  developed  a  more  appropriate  threat  description 
acronym:  ITD3.  ITD3  stands  for  Information  assurance. 
Test  conduct.  Data  quality.  Data  delivery.  Displays  & 
human  interface.  This  acronym  was  derived  from  the  use 
of  affinity  analysis  to  categorize  the  144  EMEA 
disruptions  discovered  during  the  10-week  brainstorm¬ 
ing  session.  The  Disruption  Einder  Guidebook  outlines 
the  details  of  each  category. 

Armed  with  the  ITD3  process,  the  Team  went  back 
to  the  “Fetch  Data  From  Aircraft”  scenario.  Using  the 
OSD  as  a  starting  point,  the  data-flow  diagram 
(Figure  2)  was  created. 

The  data-flow  diagram  was  analyzed  for  potential 
disruptions  using  the  ITD3  acronym.  In  addition  to 
the  previously  identified  144  potential  disruptions,  this 
process  uncovered  an  additional  61.  And,  this  process 
only  took  a  few  days  vice  10  weeks!  These  additional 
disruptions  were  also  documented  in  an  EMEA.  The 
Team  felt  they  had  a  winner.  The  process  was  not  only 
much  quicker,  but  also  more  thorough. 

Implement 

In  the  implement  phase,  the  Team  continued  to  vet  the 
new  process.  The  iNET  Disruption  Finder  Guidebook 
was  perfected.  New  threat  modeling  concepts  and  lessons 
learned  using  the  threat  models  were  incorporated.  In 
addition,  the  process  was  tested  against  the  next  three 
scenarios.  A  screen  shot  of  the  data-flow  diagram  for  one 
of  these  scenarios  (“Provide  Lossless  Telemetry  Com- 
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munications  and  Detect  Telemetry  Dropouts”)  is  shown 
in  Figure  3.  Applying  the  process  to  the  three  additional 
scenarios  yielded  51  more  potential  dismptions  that 
required  mitigation  prior  to  deploying  iNET.  Upon 
completion  of  the  threat  modeling,  the  Team  had  a 
guidebook  vetted  through  three  scenarios,  multiple 


competency  SMEs  and  process  owners,  yielding  poten¬ 
tial  plans  for  implementing  the  top  three  super-scenarios 
at  NAWCAD. 

The  Team  also  designed  a  process  map  (Figure  4) 
detailing  the  new  iNET  Deployment  process  for  users 
in  the  future. 


Critical  PCM  data 
Downloaded  data 


I 

OSD  17 

Figure  3.  “Provide  Lossless  Telemetry  Communications  (aircraft)” /“Detect  Telemetry  Drop-outs.' 
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Figure  4.  “iNET Depioyment  Process  Map.” 


Recommendations 

Once  the  AIRSpeed  Team  had  a  solid  process  for 
deploying  iNET  at  NAWCAD,  the  question  of 
process  administration  was  addressed.  The  Team  felt 
that  an  empowered  iNET  Steering  Committee  was 
required.  The  iNET  Steering  Committee  would  have 
responsibility  for  the  execution  of  the  process  described 
in  the  iNET  Disruption  Finder  Guidebook.  The  Team 
recommended  that  the  iNET  Steering  Committee  be 
created  and  empowered  by  a  Memorandum  of 
Understanding  (MOU)  signed  at  the  highest  levels  of 
the  organization  (Senior  Executive  Service  (SES) 
Level  leadership  of  the  NAVAIR  Ranges,  Flight  Test, 
and  Laboratories).  To  aid  the  functioning  of  the  iNET 
Steering  Committee,  the  Team  created  a  deployment 
CONORS.  The  iNET  Steering  Committee  is  to  meet 
regularly  with  duties  that  include  the  following: 

•  having  overall  responsibility  to  prepare  NAW¬ 
CAD  for  the  deployment  of  iNET  Technology; 

•  providing  guidance  to  users  of  the  iNET 
Disruption  Finder  Guidebook; 

•  creating  ad  hoc  SME  teams  to  brainstorm 
disruptions; 

•  assigning  ownership  and  responsibility  for  miti¬ 
gation  of  disruptions; 

•  tracking  the  mitigation  status  of  all  potential 
disruptions  to  the  deployment  of  iNET. 

Technology 

At  the  conclusion  of  the  project,  the  Team  delivered 
the  following  solution  package  to  the  project  sponsor: 


1.  iNET  Disruption  Finder  Guidebook — fa  detailed 
guide  for  creating  and  using  threat  models); 

2.  iNET  CONORS  document — (how  the  guidebook 
and  committee  work); 

3.  iNET  Master  Disruption  List  (FMEA) — (con¬ 
taining  256  disruptions  with  failure  modes  and  effects); 

4.  MOU — (an  MOU  binding  cross-competencies  to 
participate  on  the  iNET  AIR  5.0  Steering  Committee); 

5.  iNET  AIR  5.0  Steering  Committee  Charter; 

6.  recommendation  for  iNET  Steering  Committee 
first  agenda  &  schedule;  and 

7.  recommendation  for  SMEs  by  name  for  Steering 
Committee  participation. 

Implementation  status 

The  project  sponsor  is  actively  pursuing  full 
implementation  of  all  the  recommendations  from  the 
iNET  AIRSpeed  Team.  The  SES  Level  leadership  of 
the  NAVAIR  Ranges,  Flight  Test,  and  Laboratories 
have  agreed  to  the  MOU.  □ 
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